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STATUS OF CLAIMS 
Claims 1-91 are pending in the present application. Claims 1-12, 29, 36-56, 73, 80-87, 90 
and 91 stand rejected under 35 U.S.C. § 102(e). Claims 13-28, 30-35, 57-72, 74-79, 88 and 89 stand 
rejected under 35 U.S.C. § 103(a). Claims 1-91 are appealed. 

STATUS OF AMENDMENTS 
There have been no amendments filed subsequent to the final rejection. 

SUMMARY OF CLAIMED SUBJECT MATTER 
Independent claim 1 recites a method of generating an application accessible by a user 
through one or more computer-based devices, comprising the steps of: representing interactions that 
the user is permitted to have with the one or more computer-based devices used to access the 
application by interaction-based programming components, wherein the interaction-based 
programming components are independent of content/application logic and presentation 
requirements associated with the application, and further wherein the interaction-based programming 
components may be transcoded on a component by component basis to generate one or more 
modality-specific renderings of the application renderable in accordance with one or more modality- 
specific browsers associated with the one or more computer-based devices, such that the interaction- 
based programming components are independent of any modality and any modality-specific browser; 
and authoring the application using at least a portion of the interaction-based programming 
components. 

The present specification provides an illustrative embodiment of the elements of claim 1 at 
page 23, line 3 through page 25, line 26. That is, page 23, line 3 through page 25, line 26 illustrate 
one embodiment of the "representing" and "authoring" steps of claim 1. 

More particularly, FIG. 3 illustrates an example of a CML authored application according 
to an embodiment of the present invention. The CML code for generating this page is illustrated in 
FIG. 3 as CML code 10. The page fundamentally comprises a sequence of conversational gestures 
The page may comprise the following: 
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(1) Title (shown as "gesture" 20 in FIG. 3): " Global Cafe" (i.e., a particular message to be 
rendered as a Title) 

(2) A gesture message (shown as "gesture" 22 in FIG. 3): Would you like coffee, tea, milk 
or nothing? 

(3) A gesture exclusive select out of a list (shown as "gesture" 24 in FIG. 3): the list is 
composed of the following items: coffee, tea, milk and nothing. 

(4) A submit gesture (not expressly shown in FIG. 3). 

The selecting out of a list, composed of coffee, tea, milk and nothing, and submitting the 
selection is representative of interactions that the user is permitted to have with the one or more 
computer-based devices used to access the application by interaction-based programming 
components. Clearly, the page fully defines the complete interaction with the user without 
introducing any dependency on the target modality (i.e., type of access channel or access device). 
It also clearly illustrates the programming model of programming by interaction: 

(i) The application is programmed by interaction: using elementary components of 
interaction, independently of the target modality: 

(a) A gesture message: "Global Cafe." 

(b) A gesture message: Would you like coffee, tea, milk or nothing? 

(c) A gesture exclusive select out of a list. 

(d) A submit gesture. 

(ii) This is connected to the backend which is programmed/developed conventionally. In this 
example, the connection to the backend is illustrated by the list (coffee, tea, milk and nothing) that 
has been read in the backend database and added as argument to the list, either statically, when the 
page has been produced, or dynamically, when the pages have been dynamically generated on the 
server using backend logic. 
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(iii) At this stage, if needed, constraints, validations of the attributes/variables can be added, 
for example using the XFORM syntax. For example, if the page asks for the age of the user in order 
to offer alcoholic beverages, a constraint can easily be expressed that restricts or modifies the dialog 
if the users indicates that he is under age. This is not explicitly shown on the page. 

(iv) The interaction-based programming components are independent of content/application 
logic and presentation requirements associated with the application. The presentation can thereafter 
be cosmetized. In the present example, it is done by using the gesture title instead of a gesture 
message: modality independent cosmetization. Furthermore, the interaction-based programming 
components may be transcoded on a component by component basis to generate one or more 
modality-specific renderings of the application renderable in accordance with one or more modality- 
specific browsers associated with the one or more computer-based devices, such that the interaction- 
based programming components are independent of any modality and any modality- specific browser. 
Modality specific cosmetization can also be added, for example by adding HTML tags that specify 
the background (color or image) to use for the resulting HTML page. This will be ignored by the 
other target modalities or replaced by a "behavior" provided for the other modalities. For example, 
when an image is displayed in the HTML modality, a caption may be provided to be a rendering 
instead for the WML, VoiceXML, or other modalities. 

(v) The resulting pages can now be rendered by appropriate browsers. Two models exist. 
Either CML pages are served to browsers that can parse and render CML content (see Case B below) 
or they are served to legacy browsers that can only handle legacy languages, e.g., HTML, WML, 
VoiceXML, etc. (see Case A below). 

(vi) In both cases, the CML application developer does not need to do anything. He/she can 
always assume that the platform/browser will appropriately handle the rendering. 

(vii) The gestures are completely independent of the target modality. They depend also only 
on the gesture not on the backend business logic/domain or anything else. This is why the XSL rules 
can be stored on the browser, 

(viii) The XSL rules render the gestures based on the target modality. In the present case this 

means: 
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(a) Title: 

- HTML: Bold, Header character displayed 

- WML: Single card display 

- VoiceXML: Welcoming prompt 

(b) Message: 

- HTML: display in regular characters 

- WML: display in regular character (possibly on multple cards) 

- VoiceXML: Generate a prompt message (text-to-speech or play back) 

(c) Exclusive selection out of list: 

- HTML: Pull Down Menu 

- WML: Radio buttons 

- VoiceXML: Dialog (possibly Natural Language) to select in the menu (e.g., "You have that 
many items to select from. I will read the first three. Please select an item or say more for the next 
3..."). (Specification, page 23, line 3 through page 25, line 26). 

Independent claim 44, an apparatus claim having similar elements as the above-described 
claim 1, recites the additional step of obtaining the application from an application server. The 
present specification provides an illustrative embodiment of the elements of claim 44 at page 115, 
lines 4-14. More particularly, as explained therein, FIG. 14 (with continued reference to FIG. 13), 
refers to a more detailed flow diagram illustrating the operation of a multimodal browser according 
to one embodiment of the invention. An application developer writes an application, e.g., a light- 
weight application referred to as infoware, in CML. Infoware authored in CML is hosted by the a 
conversational shell (e.g., multimodal shell 62 of FIG. 13) that mediates amongst multiple modality 
specific browser components (e.g., visual browser 64 and speech browser 66 of FIG. 13). The 
multimodal shell maybe thought of as a CML interpreter or processor. This is illustrated in FIG. 
14 as block 70. User interaction proceeds by the CML interpreter mapping CML instances 
associated with the downloaded CML code to appropriate modality-specific languages such as 
HTML (block 77) and VoiceXML (block 78). (Specification, page 115, lines 4-14). Based on the 
user's request, the multimodal browser 60 sends an appropriate URL to a content server 69 (an 
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example of the claimed step of obtaining the application from an application server), which also 
services conversational engines 68 that may also reside on the client device, in order to request 
access to the particular desired application. To obtain the application from content server 69, CML 
code associated with the application is then downloaded from the content server 69 to the 
multimodal browser 60. The multimodal browser then generates the modality specific renderings 
(GUI representation and/or speech representation) based on the conversational gestures associated 
with the CML code. The user thus interacts with the browser 60 via these representations. 
(Specification, page 114, line 24 through page 115, line 3). 

Independent claim 90 is a browser apparatus claim having similar elements as the above- 
described claim 44. The browser apparatus comprises a machine readable medium. The present 
specification provides an illustrative embodiment of the elements of claim 90 at page 1 1 9, lines 2-6. 
More particularly, as explained therein, software components including instructions or code for 
performing the methodologies of the invention, as described herein, may be stored in one or more 
of the associated memory devices (e.g., ROM, fixed or removable memory) and, when ready to be 
utilized, loaded in part or in whole (e.g., into RAM) and executed by a CPU. (Specification, page 
119, lines 2-6). 

Independent claim 91 is an article of manufacture claim having similar elements as the 
above- described claim 1. The article of manufacture comprises a machine readable medium. The 
present specification provides an illustrative embodiment of the elements of claim 91 at pagell9, 
lines 2-6. More particularly, as explained therein, software components including instructions or 
code for performing the methodologies of the invention, as described herein, may be stored in one 
or more of the associated memory devices (e.g., ROM, fixed or removable memory) and, when ready 
to be utilized, loaded in part or in whole (e.g., into RAM) and executed by a CPU. (Specification, 
page 119, lines 2-6). 

GROUNDS OF REJECTION TO BE REVIEWED ON APPEAL 
(1) Claims 1-12, 29, 36-56, 73, 80-87, 90 and 91 are rejected under 35 U.S.C. § 102(e) as 
being anticipated by U.S. Patent No. 6,418,439 to Papierniak et al. (hereinafter "Papiemiak"). 



6 



Attorney Docket No. YQ999-478 

(2) Claims 5 and 49 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Papierniak in view of "New VXML Forum," posted at Cover Pages Hosted by Oasis (hereinafter 
"New VXML Forum"). 

(3) Claims 13-28, 34, 57-72 and 78 are rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Papierniak in view of U.S. Patent No. 6,269,336 to Ladd et al. (hereinafter 
"Ladd"). 

(4) Claims 3 1 , 32, 75 and 76 are rejected under 35 U.S.C. § 1 03(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,569,207 to Sundarsesan (hereinafter "Sundarsesan"). 

(5) Claims 30 and 74 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Papierniak in view of World Wide Web Consortium document entitled "Extensible Stylesheet 
Language (XSL) version 1 .0" (hereinafter referred to as "W3C XSL specification"). 

(6) Claims 33, 77, and 88 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,493,758 to McLain (hereinafter "McLain"). 

(7) Claims 35, 79, and 89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,456,974 to Baker et al. (hereinafter "Baker"). 

ARGUMENT 

(1) Claims 1-12, 29, 36-56, 73, 80-87, 90 and 91 are rejected under 35 U.S.C. §102(e) as 
being anticipated by U.S. Patent No. 6,418,439 to Papierniak et al. (hereinafter "Papierniak"). 

Regarding the § 102(e) rejections of claims 1-12, 29, 36-56, 73, 80-87, 90 and 91, Appellants 
assert that Papierniak fails to teach or suggest all of the limitations in said claims for at least the 
reasons presented below. 

It is well-established law that a claim is anticipated only if each and every element as set forth 
in the claim is found, either expressly or inherently described, in a single prior art reference. 
Verdegaal Bros. v. Union Oil Co. of California, 814 F.2d 628, 631, 2U.S.P.Q.2d 1051, 1053 (Fed. 
Cir. 1987). Appellants assert that the rejection based on Papierniak does not meet this basic legal 
requirement, as will be explained below. 



7 



Attorney Docket No. YQ999-478 

The present invention, for example, as recited in amended independent claim 1, recites a 
method of generating an application accessible by a user through one or more computer-based 
devices, comprising the following steps. Interactions that the user is permitted to have with the one 
or more computer-based devices used to access the application are represented by interaction-based 
programming components. The interaction-based programming components are independent of 
content/application logic and presentation requirements associated with the application. Further, the 
interaction-based programming components maybe transcoded on a component by component basis 
to generate one or more modality-specific renderings of the application renderable in accordance 
with one or more modality-specific browsers associated with the one or more computer-based 
devices, such that the interaction-based programming components are independent of any modality 
and any modality- specific browser . The application is then authored using at least a portion of the 
interaction-based programming components. Independent claim 91 recites similar limitations in an 
article of manufacture format. Independent claims 44 and 90 recite respective apparatus for 
accessing applications having similar limitations. 

Accordingly, as explained above in the Summary section, a new paradigm is introduced, 
illustratively embodied as a Conversational Markup Language (CML), which provides for separating 
application programming into content aspects, presentation aspects and interaction aspects. By 
focusing on the interaction aspect of an application with respect to a user, an application may be 
written in a manner which is independent of the content/application logic and presentation. The 
"interaction-based programming components" recited in independent claims 1 , 44, 90 and 91 provide 
such advantages since, as expressly recited, they are independent of content/application logic and 
presentation requirements associated with the application and they may be transcoded on a 
component by component basis to generate one or more modality-specific renderings of the 
a pplication renderable in accordance with one or more modality-specific browsers associated with 
the one or more computer-based devices, such that the interaction-based programming components 
are independent of any modality and any modality-specific browser . Examples of such presentation 
requirements and modality-specific renderings include visual-based (e.g., text and graphical) 
renderings, speech based renderings, and combinations thereof. 
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Papierniak is significantly different than the claimed invention. Papierniak discloses at 
column 9, lines 25-53: 



FIG. 6 depicts a flow chart for the translation of information into multiple media variations 
with different views for a distributed on-line, and interactive environment in accordance with 
a second embodiment of the invention. In FIG. 6, the Web Synchronizer receives View A 
which is text in Step S30. View A can be received, for example, as text, and either stored as 
text and translated when requested, or translated immediately into views and then stored. In 
Step S32, each kind of user is assigned a user type, e.g., User 1, User 2, User 3, User 4. In 
Step S34, the presentation is set in the Web Synchronizer/Translator. Each user is prompted 
for or assigned a view, e.g., Video-User 1, Audio-User 2 Graphics-User 3, Text-User 4, 
or a combination of these views. 

A Person logs-on and is assigned or selects a user type, e.g., User 1 , in Step S36, and a user, 
for example, User 1 requests View A in Step S38. View A is converted to, for example, 
video and User 1 receives the video presentation of View A in Step S40. In Step S42, a 
second person that may log-on the Web server is assigned or selects a user type, e.g., User 
2, User 2 requests View A in Step S44, and the Web synchronizer/translator converts or 
retrieves View A to Audio which is then received by User 2 as an Audio Presentation of 
View A in Step S46. The process then continues on with other user request in Step S48, and 
control is returned to Step S36. Note that the translation/conversion among multiple media 
variations can occur either on-line or off-line, at either the information source(s) or web 
synchronizer/translator. 

Thus, while Papierniak describes different "views" as being assigned to different users, these 
"views" correspond to modality-specific data, i.e., audio, graphic, and text. Such data is provided 
by modality- specific information sources. That is, column 8, lines 53-55 , of Papierniak explains that 
"[t]he components of these information sources can be text-based, audio-based, video-based, image- 
based, graphic-based, and, in general, multimedia-based." Thus, unlike the claimed invention, no 
where does Papierniak disclose that "interaction-based programming components are independent 
of any modality and any modality- specific browser . 

Even in the embodiment of FIG. 7 of Papierniak where it appears that text data is processed 
by a "web presence synchronizer," the initial information from the sources is still in the modality- 
specific form of text. That is, while Papierniak discloses a technique for translation of information 
into multiple media variations, the information before the translation is still in a modality-specific 
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(text) form, and is merely translated to another modality-specific format (audio, graphic, etc.). Thus, 
it cannot be concluded that the data or any programming components described in Papiemiak are 
independent of any modality, as in the claimed invention. 

For at least the above reasons, Appellants assert that independent claims 1 , 44, 90 and 91 are 
patentable over Papierniak. 

Further, Appellants assert that dependent claims 2-12, 29, 36-43, 45-56, 73 and 80-87 are 
patentable over Papierniak not only because they respectively depend from independent claims 1 
and 44, but also because said claims recite patentable subject matter in their own right. 

By way of example only, claims 8 and 52 recite wherein the user interactions are 
declarativelyrepresentedby the interaction-based programming components. Claims 9 and 53 recite 
wherein the user interactions are imperatively represented by the interaction-based programming 
components. Claims 10 and 54 recite wherein the user interactions are declaratively and 
imperatively represented by the interaction-based programming components. Claims 11 and 55 
recite wherein the interaction-based programming components comprise basic elements associated 
with a dialog that may occur between the user and the one or more computer-based devices. Claims 
12 and 56 recite wherein the interaction-based programming components comprise complex 
elements, the complex elements being aggregations of two or more of the basic elements associated 
with the dialog that may occur between the user and the one or more computer-based devices. The 
Office Action repeatedly cites column 9, lines 25-53, of Papierniak (copied in its entirety and 
addressed above) in rejecting the claims. However, nowhere can Appellants find any such features 
disclosed therein. 

Similarly, claims 80 through 87 are respectively rejected in the Office Action by citation to 
column 9, lines 25-53, of Papierniak. However, nowhere does Papierniak therein disclose the 
claimed features. 

Accordingly, withdrawal of the § 102(e) rejections is respectfully requested. 
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(2) Claims 5 and 49 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Papierniak in view of "New VXML Forum," posted at Cover Pages Hosted by Oasis (hereinafter 
"New VXML Forum"). 

Regarding the §103(a) rejections of claims 5 and 49 under 35 U.S.C. § 103(a) based on 
combination of Papierniak and the New VXML Forum, Appellants assert that said claims are 
patentable over the combination not only because they respectively depend from independent claims 
1 and 44, but also because said claims recite patentable subject matter in their own right. 

Further, there is a clear lack of motivation to combine the references. The Federal Circuit 
has stated that when patentability turns on the question of obviousness, the obviousness 
determination "must be based on objective evidence of record" and that "this precedent has been 
reinforced in myriad decisions, and cannot be dispensed with." In re Lee, 277F.3d 1338, 1343 (Fed. 
Cir. 2002). Moreover, the Federal Circuit has stated that "conclusory statements" by an examiner 
fail to adequately address the factual question of motivation, which is material to patentability and 
cannot be resolved "on subjective belief and unknown authority." Id. at 1 343-1 344. 

In the Office Action at page 6, the Examiner provides the following statement to prove 
motivation to combine Papierniak and the New VXML Forum, with emphasis supplied: "[i]t would 
have been obvious for one of ordinary skill in the Computer Networking art at the time of the 
invention to combine the teachings of Papierniak regarding a speech applications system with 
VoiceXML because VoiceXML is a form of SGML document ." 

Appellants submit that this statement is based on the type of "subjective belief and unknown 
authority" that the Federal Circuit has indicated provides insufficient support for an obviousness 
rejection. More specifically, the Examiner fails to identify any objective evidence of record which 
supports the proposed combination. 

Accordingly, withdrawal of the §1 03(a) rejections is respectfully requested. 

(3) Claims 13-28, 34, 57-72 and 78 are rejected under 35 U.S.C. § 103(a) as being 
unpatentable over Papierniak in view of U.S. Patent No. 6,269,336 to Ladd et al. (hereinafter 
"Ladd"). 
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Regarding the § 103(a) rejections of claims 13-28, 34, 57-72 and 78 under 35 U.S.C. § 103(a) 
based on combination of Papierniak and Ladd, Appellants assert that said claims are patentable over 
the combination not only because they respectively depend from independent claims 1 and 44, but 
also because said claims recite patentable subject matter in their own right. 

By way of example, claims 13-28 and 57-72 recite various detailed features of the 
conversational gestures that make up the language of the invention. The Office Action cites code 
in Ladd at column 12, lines 30-67, in rejecting the majority of the claims. The other claims are 
rejected based on column 18, lines 56-65. However, column 12, lines 30-67, discloses "syntax and 
grammar that the parser unit of the voice browser utilizes to build a tree structure of the markup 
language document." None of this code, nor anything at column 18, lines 56-65, discloses the 
features of the claimed conversational gestures of the invention. 

Further, based on the rationale given by the Examiner at page 7, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the §103 (a) rejections is respectfully requested. 

(4) Claims 31,32,75 and 76 are rejected under 35 U.S.C. § 1 03(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,569,207 to Sundarsesan (hereinafter "Sundarsesan"). 

Regarding the § 1 03(a) rejections of claims 3 1 , 32, 75 and 76 under 35 U.S.C. § 1 03(a) based 
on combination of Papierniak and Sundarsesan, Appellants assert that said claims are patentable over 
the combination not only because they respectively depend from independent claims 1 and 44, but 
also because said claims recite patentable subject matter in their own right. 

Further, based on the rationale given by the Examiner at page 9, there is a clear lack of 
motivation to combine the references based on the rationale of in re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

(5) Claims 30 and 74 are rejected under 35 U.S.C. § 103(a) as being unpatentable over 
Papierniak in view of World Wide Web Consortium document entitled "Extensible Stylesheet 
Language (XSL) version 1.0" (hereinafter referred to as "W3C XSL specification"). 



12 



Attorney Docket No. YQ999-478 

Regarding the § 103(a) rejections of claims 30 and 74 under 35 U.S.C. § 103(a) based on 
combination of Papierniak and W3C XSL specification, Appellants assert that said claims are 
patentable over the combination not only because they respectively depend from independent claims 
1 and 44, but also because said claims recite patentable subject matter in their own right. 

Further, based on the rationale given by the Examiner at page 10, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 

(6) Claims 33, 77 and 88 are rejected under 35 U.S.C. §103(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,493,758 to McLain (hereinafter "McLain"). 

Regarding the § 103(a) rejections of claims 33, 77 and 88 under 35 U.S.C. § 103(a) based on 
combination of Papierniak and McLain, Appellants assert that said claims are patentable over the 
combination not only because they respectively depend from independent claims 1 and 44, but also 
because said claims recite patentable subject matter in their own right. 

Claims 33 and 77 recite wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality- specific renderings of the 
application on the one or more computer-based devices. Claim 88 recites wherein the one or more 
processors are distributed over the one or more computer-based devices and the application is 
synchronized across the one or more computer-based devices. 

The only synchronization that McLain discloses (column 3, line 50, through column 4, line 
6) is that of synchronizing information between a desktop computer 1 6 and amobile device 18, This 
is not unlike "hot-syncing" a personal digital assistant (PDA) with a desktop so that the PDA has the 
up-to-date data from the desktop. This is not the same as representation by the interaction-based 
programming components permitting synchronization of the one or more modality-specific 
renderings of the application on the one or more computer-based devices, as recited in claims 33 and 
77, or the one or more processors being distributed over the one or more computer-based devices and 
the application being synchronized across the one or more computer-based devices, as recited in 
claim 88. 
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Further, based on the rationale given by the Examiner at page 11, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 
Accordingly, withdrawal of the §103(a) rejections is respectfully requested. 

(7) Claims 35, 79, and 89 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Papierniak in view of U.S. Patent No. 6,456,974 to Baker et al. (hereinafter "Baker"). 

Regarding the § 103(a) rejections of claims 35, 79, and 89 under 35 U.S.C. § 103(a) based on 
combination of Papierniak and Baker, Appellants assert that said claims are patentable over the 
combination not only because they respectively depend from independent claims 1 and 44, but also 
because said claims recite patentable subject matter in their own right. 

Claims 35 and 79 recite the step of including code in the authored application for permitting 
cosmetic altering of a presentational feature associated with the one or more modality-specific 
renderings of the application on the one or more computer-based devices. Claim 89 recites wherein 
the representation of the application further permits cosmetization of the one or more modality- 
specific renderings via one or more modality-specific markup languages. 

The Office Action cites column 3, lines 7-32, of Baker. However, nowhere therein does 
Baker mention the claimed cosmetization features. The only things mentioned in Baker therein are 
"scroll-up" and "go back" commands. 

Further, based on the rationale given by the Examiner at page 11, there is a clear lack of 
motivation to combine the references based on the rationale of In re Lee, as cited above. 

Accordingly, withdrawal of the § 103(a) rejections is respectfully requested. 
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In view of the above, Appellants believe that claims 1 -91 are in condition for allowance, and 
respectfully request favorable reconsideration. 



Respectfully submitted, 



Attorney for Appellant($lJ 
Reg. No. 39,274 
Ryan, Mason & Lewis, LLP 
90 Forest Avenue 
Locust Valley, NY 11560 
(516) 759-2946 



Date: May 14, 2007 




15 



Attorney Docket No. YQ999-478 

CLAIMS APPENDIX 

1 . A method of generating an application accessible by a user through one or more computer- 
based devices, the method comprising the steps of: 

representing interactions that the user is permitted to have with the one or more computer- 
based devices used to access the application by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application, and further wherein the interaction-based 
programming components may be transcoded on a component by component basis to generate one 
or more modality-specific renderings of the application renderable in accordance with one or more 
modality-specific browsers associated with the one or more computer-based devices, such that the 
interaction-based programming components are independent of any modality and any modality- 
specific browser; and 

authoring the application using at least a portion of the interaction-based programming 
components. 

2. The method of claim 1, in a client/server arrangement wherein at least a portion of the 
application is to be downloaded from a server to at least one of the one or more computer-based 
devices, acting as a client, further comprising the step of including code in the application operative 
to provide a connection to the content/application logic resident at the server. 

3. The method of claim 2, wherein the code in the application operative to provide a 
connection to the content/application logic expresses at least one of one or more data models, 
attribute constraints and validation rules associated with the application. 

4. The method of claim 1, wherein the one or more modality-specific renderings comprise 
a speech-based representation of portions of the application. 
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5 . The method of claim 4, wherein the speech-based representation is based on VoiceXML. 

6. The method of claim 1, wherein the one or more modality-specific renderings comprise 
a visual-based representation of portions of the application. 

7. The method of claim 6, wherein the visual-based representation is based on at least one 
of HTML, CHTML and WML. 

8. The method of claim 1 , wherein the user interactions are declaratively represented by the 
interaction-based programming components. 

9. The method of claim 1, wherein the user interactions are imperatively represented by the 
interaction-based programming components. 

1 0. The method of claim 1 , wherein the user interactions are declaratively and imperatively 
represented by the interaction-based programming components. 

11. The method of claim 1, wherein the interaction-based programming components 
comprise basic elements associated with a dialog that may occur between the user and the one or 
more computer-based devices. 

12. The method of claim 11, wherein the interaction-based programming components 
comprise complex elements, the complex elements being aggregations of two or more of the basic 
elements associated with the dialog that may occur between the user and the one or more computer- 
based devices. 

13. The method of claim 1 , wherein one of the interaction-based programming components 
represent conversational gestures. 
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14. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating informational messages to the user. 

15. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating contextual help information. 

16. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating actions to be taken upon successful completion of another gesture. 

17. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating yes or no based questions. 

18. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues where the user is expected to select from a set of choices. 

19. The method of claim 18, wherein the select gesture comprises a subelement that 
represents the set of choices. 

20. The method of claim 18, wherein the select gesture comprises a subelement that 
represents a test that the selection should pass. 

21. The method of claim 20, wherein the select gesture comprises a subelement that 
represents an error message to be presented if the test fails. 

22. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating rules for validating results of a given conversational gesture. 
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23. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating grammar processing rules. 

24. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues that help the user navigate through portions of the application. 

25. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for at least one of user login and authentication information. 

26. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for constrained user input. 

27. The method of claim 13, wherein the conversational gestures comprise a gesture for 
encapsulating a request for unconstrained user input. 

28. The method of claim 13, wherein the conversational gestures comprise a gesture for 
controlling submission of information. 

29. The method of claim 1, further comprising the step of defining logical input events and 
an association between the logical input events and physical input events that trigger the defined 
logical input events, such that the application may be authored using at least a portion of the 
definitions. 

30. The method of claim 1, wherein the component by component transcoding is performed 
in accordance with XSL transformation rules. 

3 1 . The method of claim 1 , wherein the component by component transcoding is performed 
in accordance with Java Bean. 
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32. The method of claim 1, wherein the component by component transcoding is performed 
in accordance with Java Server Pages. 

33. The method of claim 1, wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality-specific renderings of the 
application on the one or more computer-based devices. 

34. The method of claim 1, wherein representation by the interaction-based programming 
components supports a natural language understanding environment. 

35. The method of claim 1, further comprising the step of including code in the authored 
application for permitting cosmetic altering of a presentational feature associated with the one or 
more modality- specific renderings of the application on the one or more computer-based devices. 

36. The method of claim 1, further comprising the step of including code in the authored 
application for permitting changes to rules for transcoding on a component by component basis to 
generate the one or more modality-specific renderings of the application on the one or more 
computer-based devices. 

3 7 . The method of claim 1 , wherein adefinitionofan underlying data model being populated 
is separated from a markup language defining the user interaction. 

38. The method of claim 1, wherein a node_id attribute is attached to each component and 
the attribute is mapped over to various outputs. 

39. The method of claim 1 , wherein an author is provided with a pass through mechanism 
to encapsulate modality-specific markup components. 
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40. The method of claim 1, wherein the components maybe active in parallel. 

41. The method of claim 1, wherein the representation and transcoding is extensible. 

42. The method of claim 1 , wherein a state of the application is encapsulated. 

43. The method of claim 1, wherein the representation permits reference to dynamically 
generated data and supports callback mechanisms to the content/application logic. 

44. Apparatus for use in accessing an application in association with one or more computer- 
based devices, the apparatus comprising: 

one or more processors operative to: (i) obtain the application from an application server, the 
application being pro grammatically represented by interactions that the user is permitted to have with 
the one or more computer-based devices by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application; and (ii) transcode the interaction-based 
programming components on a component by component basis to generate one or more modality- 
specific renderings of the application renderable in accordance with one or more modaility-specific 
browsers associated with the one or more computer-based devices, the interaction-based 
programming components being independent of any modality and any modality-specific browser. 

45. The apparatus of claim 44, wherein the one or more processors are distributed over the 
one or more computer-based devices. 

46. The apparatus of claim 44, in a client/server arrangement wherein at least a portion of 
the application is to be downloaded from a server to at least one of the one or more computer-based 
devices, acting as a client, further comprising the step of including code in the application operative 
to provide a connection to the content/application logic resident at the server. 
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47. The apparatus of claim 46, wherein the code in the application operative to provide a 
connection to the content/application logic expresses at least one of one or more data models, 
attribute constraints and validation rules associated with the application. 

48. The apparatus of claim 44, wherein the one or more modality-specific renderings 
comprise a speech-based representation of portions of the application. 

49. The apparatus of claim 48, wherein the speech-based representation is based on 
VoiceXML. 

50. The apparatus of claim 44, wherein the one or more modality- specific renderings 
comprise a visual-based representation of portions of the application. 

51. The apparatus of claim 50, wherein the visual-based representation is based on at least 
one of HTML, CHTML and WML. 

52. The apparatus of claim 44, wherein the user interactions are declaratively represented 
by the interaction-based programming components. 

53 . The apparatus of claim 44, wherein the user interactions are imperatively represented by 
the interaction-based programming components. 

54. The apparatus of claim 44, wherein the user interactions are declaratively and 
imperatively represented by the interaction-based programming components. 

55. The apparatus of claim 44, wherein the interaction-based programming components 
comprise basic elements associated with a dialog that may occur between the user and the one or 
more computer-based devices. 
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56. The apparatus of claim 55, wherein the interaction-based programming components 
comprise complex elements, the complex elements being aggregations of two or more of the basic 
elements associated with the dialog that may occur between the user and the one or more computer- 
based devices. 

57. The apparatus of claim 44, wherein one of the interaction-based programming 
components represent conversational gestures. 

58. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating informational messages to the user. 

59. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating contextual help information. 

60. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating actions to be taken upon successful completion of another gesture. 

61. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating yes or no based questions. 

62. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues where the user is expected to select from a set of choices. 

63. The apparatus of claim 62, wherein the select gesture comprises a subelement that 
represents the set of choices. 

64. The apparatus of claim 62, wherein the select gesture comprises a subelement that 
represents a test that the selection should pass. 
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65. The apparatus of claim 64, wherein the select gesture comprises a subelement that 
represents an error message to be presented if the test fails. 

66. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating rules for validating results of a given conversational gesture. 

67. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating grammar processing rules. 

68. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating dialogues that help the user navigate through portions of the application. 

69. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for at least one of user login and authentication information. 

70. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for constrained user input. 

71. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
encapsulating a request for unconstrained user input. 

72. The apparatus of claim 57, wherein the conversational gestures comprise a gesture for 
controlling submission of information. 

73. The apparatus of claim 44, further comprising the step of defining logical input events 
and an association between the logical input events and physical input events that trigger the defined 
logical input events, such that the application may be authored using at least a portion of the 
definitions. 
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74. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with XSL transformation rules. 

75. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with Java Bean. 

76. The apparatus of claim 44, wherein the component by component transcoding is 
performed in accordance with Java Server Pages. 

77. The apparatus of claim 44, wherein representation by the interaction-based programming 
components permits synchronization of the one or more modality- specific renderings of the 
application on the one or more computer-based devices. 

78. The apparatus of claim 44, wherein representation by the interaction-based programming 
components supports a natural language understanding environment. 

79. The apparatus of claim 44, further comprising the step of including code in the authored 
application for permitting cosmetic altering of a presentational feature associated with the one or 
more modality-specific renderings of the application on the one or more computer-based devices. 

80. The apparatus of claim 44, further comprising the step of including code in the authored 
application for permitting changes to rules for transcoding on a component by component basis to 
generate the one or more modality- specific renderings of the application on the one or more 
computer-based devices. 

81. The apparatus of claim 44, wherein a definition of an underlying data model being 
populated is separated from a markup language defining the user interaction. 
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82. The apparatus of claim 44, wherein a node_id attribute is attached to each component 
and the attribute is mapped over to various outputs. 

83 . The apparatus of claim 44, wherein an author is provided with a pass through mechanism 
to encapsulate modality-specific markup components. 

84. The apparatus of claim 44, wherein the components may be active in parallel. 

85. The apparatus of claim 44, wherein the representation and transcoding is extensible. 

86. The apparatus of claim 44, wherein a state of the application is encapsulated. 

87. The apparatus of claim 44, wherein the representation permits reference to dynamically 
generated data and supports callback mechanisms to the content/application logic. 

88. The apparatus of claim 44, wherein the one or more processors are distributed over the 
one or more computer-based devices and the application is synchronized across the one or more 
computer-based devices. 

89. The apparatus of claim 44, wherein the representation of the application further permits 
cosmetization of the one or more modality-specific renderings via one or more modality-specific 
markup languages. 

90. A browser apparatus for use in providing access to an application by a user through one 
or more computer-based devices, comprising a machine readable medium containing computer 
executable code which when executed permits the implementation of the steps of: 

obtaining the application from an application server, the application being programmatically 
represented by interactions that the user is permitted to have with the one or more computer-based 



26 



Attorney Docket No. YQ999-478 

devices by interaction-based programming components, wherein the interaction-based programming 
components are independent of content/application logic and presentation requirements associated 
with the application; and 

transcoding the interaction-based programming components on a component by component 
basis to generate one or more modality-specific renderings of the application renderable in 
accordance with one or more modaility-specific browsers associated with the one or more computer- 
based devices, the interaction-based programming components being independent of any modality 
and any modality- specific browser. 

91. An article of manufacture for use in generating an application accessible by a user 
through one or more computer-based devices, comprising a machine readable medium containing 
computer executable code which when executed permits the implementation of the steps of: 

representing interactions that the user is permitted to have with the one or more computer- 
based devices used to access the application by interaction-based programming components, wherein 
the interaction-based programming components are independent of content/application logic and 
presentation requirements associated with the application, and further wherein the interaction-based 
programming components may be transcoded on a component by component basis to generate one 
or more modality- specific renderings of the application renderable in accordance with one or more 
modality-specific browsers associated with the one or more computer-based devices, such that the 
interaction-based programming components are independent of any modality and any modality- 
specific browser; and 

authoring the application using at least a portion of the interaction-based programming 
components. 
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EVIDENCE APPENDIX 
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